Tracking system, terminal device, camera device, tracking imaging method, and program

ABSTRACT

In a preferred aspect of the present invention, at least one of a camera-side controller or a terminal-side controller performs a tracking image generation process (P 1 ) of generating tracking image data from captured image data. Further, at least one of the camera-side controller or the terminal-side controller performs a tracking calculation process (P 2 ) of acquiring target information on the basis of the tracking image data. The tracking image data includes a high-resolution region with relatively higher resolution and a low-resolution region with relatively lower resolution. In the tracking image generation process (P 1 ), basic tracking information including information on at least one of a size or a speed of the tracking target is acquired, and a high-resolution region and a low-resolution region are determined on the basis of the basic tracking information. The tracking image generation process (P 1 ) and the tracking calculation process (P 2 ) are performed in parallel.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a Continuation of PCT InternationalApplication No. PCT/JP2015/083599 filed on Nov. 30, 2015 claimingpriority under 35 U.S.C §119(a) to Japanese Patent Application No.2015-18642 filed on Feb. 2, 2015. Each of the above applications ishereby expressly incorporated by reference, in their entirety, into thepresent application.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a tracking system that acquires aposition of a tracking target on the basis of tracking image data ofwhich resolution is adjusted, a terminal device, a camera device, atracking imaging method, and a program.

2. Description of the Related Art

A tracking imaging technology for continuously performing imaging whiletracking a specific subject is widely used in the field of a monitoringcamera, a general digital camera, or the like. Various methods have beenproposed as schemes of specifying a position of a specific subjectserving as a tracking target. For example, a method of specifying theposition of the tracking target in an image by collating a feature ofthe tracking target with a captured image is known.

From the viewpoint of improvement of a recognition rate of a trackingtarget, it is preferable to specify a position of the tracking target onthe basis of the high resolution image, but generally, as resolution ofan image increases, a processing load increases. Therefore, even in acase where a position of the tracking target is specified using an imagewith high resolution, a delay occurs in an image analysis process in atracking system in which capability of a process of analyzing such ahigh resolution image is insufficient, and as a result, trackingaccuracy may be degraded. In particular, in a case where a motion of thetracking target is fast, adverse effects on a tracking process as wellas a process other than the tracking process, such as a tendency of longtime required for the process due to an increasing amount of processcalculation, and a decreasing frame rate of a captured image, areassumed.

Under such circumstances, in the imaging device with a tracking functiondisclosed in JP2014-216694A, high-resolution processing is performed ononly a tracking target object and the inside of a tracking frame formedaround the tracking target, and a position of the tracking target isspecified on the basis of an image on which the high-resolutionprocessing has been performed.

SUMMARY OF THE INVENTION

As described above, the image data suitable for a tracking process(hereinafter also referred to as “tracking image data”) is generatedfrom captured image data and the position of the tracking target isspecified on the basis of the tracking image data, making it possible toperform a tracking process adapted to a processing capacity of thetracking system.

However, in a case where the position of the tracking target isspecified on the basis of the tracking image data, a process ofgenerating the tracking image data is necessary and the number ofprocess steps increases, as compared with a case where the position ofthe tracking target is specified on the basis of the captured imagedata.

In particular, in a case where resolution of the tracking image data isadjusted according to the position of the tracking target as in theimaging device of JP2014-216694A, it is necessary to perform a “processof generating tracking image data according to a position of a trackingtarget” after a “process of specifying a position of a tracking targeton the basis of tracking image data”. Further, it is necessary toperform the “process of specifying the position of the tracking targeton the basis of the tracking image data” after the “process ofgenerating tracking image data according to the position of the trackingtarget”. Thus, in a case where the “process of specifying the positionof the tracking target” and the “process of generating the trackingimage data” are executed in series, if a delay occurs during theprocess, tracking accuracy is degraded due to an influence of the delayon the other entire process in a subsequent step.

The present invention has been made in view of the above circumstances,and an object thereof is to provide a technology capable of speeding upa process of tracking-imaging a tracking target while ensuring goodtracking accuracy.

An aspect of the present invention relates to a tracking system,comprising: a camera device including an imaging unit that acquirescaptured image data, an imaging direction adjustment unit that adjustsan imaging direction of the imaging unit, a camera-side controller thatcontrols the imaging direction adjustment unit on the basis of targetinformation indicating a position of a tracking target, and acamera-side communication unit that is connected to the camera-sidecontroller; and a terminal device including a terminal-sidecommunication unit that is able to communicate with the camera-sidecommunication unit, a terminal-side controller that is connected to theterminal-side communication unit, and a user interface that is connectedto the terminal-side controller, in which at least one of thecamera-side controller or the terminal-side controller performs atracking image generation process of generating tracking image data fromthe captured image data, at least one of the camera-side controller orthe terminal-side controller performs a tracking calculation process ofacquiring the target information on the basis of the tracking imagedata, the tracking image data is data of an image including ahigh-resolution region with relatively higher resolution and alow-resolution region with relatively lower resolution, in the trackingimage generation process, basic tracking information includinginformation on at least one of a size or a speed of the tracking targetis acquired, the high-resolution region and the low-resolution regionare determined on the basis of the basic tracking information, and thetracking image data is generated, and the tracking image generationprocess and the tracking calculation process are performed in parallel.

According to this aspect, since the tracking image data is generated onthe basis of information on at least one of the size or the speed of thetracking target, and the tracking image generation process and thetracking calculation process are performed in parallel, it is possibleto effectively speed up a process of tracking-imaging the trackingtarget while ensuring good tracking accuracy.

Preferably, at least a portion of the tracking calculation process ofacquiring the target information indicating the position of the trackingtarget at a first timing on the basis of the tracking image datagenerated from the captured image data acquired at the first timing isperformed simultaneously with at least a portion of the tracking imagegeneration process of generating the tracking image data that is used toacquire the target information indicating the position of the trackingtarget at a second timing after the first timing, from the capturedimage data acquired at the second timing.

According to this aspect, at least a portion of the tracking calculationprocess of acquiring the target information indicating the position ofthe tracking target at the first timing is performed simultaneously withat least a portion of the tracking image generation process ofgenerating the tracking image data that is used to acquire targetinformation indicating the position of the tracking target at the secondtiming, making it possible to speed up a process for tracking-imagingthe tracking target.

The “tracking calculation process of acquiring target informationindicating the position of the tracking target at the first timing” andthe “tracking image generation process of generating the tracking imagedata that is used to acquire target information indicating the positionof the tracking target at the second timing” may be started at the sametime, and one of the processes may be started after the start of theother of processes and before completion of the other of the process.

Preferably, the tracking image generation process and the trackingcalculation process are performed in one of the camera-side controllerand the terminal-side controller.

According to this aspect, the tracking image generation process and thetracking calculation process are performed in one of the camera-sidecontroller and the terminal-side controller, making it possible tosimplify a processing flow of the tracking image generation process andthe tracking calculation process.

Preferably, one of the tracking image generation process and thetracking calculation process is performed in one of the camera-sidecontroller and the terminal-side controller, and the other of thetracking image generation process and the tracking calculation processis performed in the other of the camera-side controller and theterminal-side controller.

According to this aspect, the tracking image generation process and thetracking calculation process are performed in a distributed manner bythe camera-side controller and the terminal-side controller, making itpossible to effectively utilize resources.

Preferably, the tracking calculation process is performed by both of thecamera-side controller and the terminal-side controller, and acontroller having a relatively higher processing capability between thecamera-side controller and the terminal-side controller performs thetracking calculation process of the high-resolution region, and acontroller having a relatively lower processing capability between thecamera-side controller and the terminal-side controller performs thetracking calculation process of the low-resolution region.

According to this aspect, the tracking calculation process of thehigh-resolution region and the tracking calculation process of thelow-resolution region can be performed efficiently according toprocessing capacity of the camera-side controller and the terminal-sidecontroller.

Preferably, at least one of the camera-side controller or theterminal-side controller performs a pre-search process of acquiring aspatial frequency of the captured image data of at least a portion ofthe imaging range of the camera device, and in the tracking imagegeneration process, resolution of the tracking image data is determinedon the basis of the spatial frequency acquired through the pre-searchprocess.

According to this aspect, it is possible to adaptively determine theresolution of the tracking image data on the basis of the spatialfrequency of at least a portion of the imaging range of the cameradevice, and it is possible to reduce a processing load according to animaging environment.

Preferably, in the tracking image generation process, in a case wherethe spatial frequency acquired through the pre-search process isrelatively low, the resolution of the tracking image data is set to belower than that in a case where the spatial frequency is relativelyhigh.

According to this aspect, in a case where the spatial frequency of atleast a portion of the imaging range of the camera device is relativelylow, it is possible to reduce a processing load and speed up the processof tracking-imaging the tracking target in comparison with the casewhere the spatial frequency of the at least a portion of the imagingrange is relatively high.

Another aspect of the present invention relates to a terminal devicethat is connectable to a camera device including an imaging unit thatacquires captured image data, an imaging direction adjustment unit thatadjusts an imaging direction of the imaging unit, a camera-sidecontroller that controls the imaging direction adjustment unit on thebasis of target information indicating a position of a tracking target,and a camera-side communication unit that is connected to thecamera-side controller, the terminal device comprising: a terminal-sidecommunication unit that is able to communicate with the camera-sidecommunication unit; a terminal-side controller that is connected to theterminal-side communication unit; and a user interface connected to theterminal-side controller, in which at least one of the camera-sidecontroller or the terminal-side controller performs a tracking imagegeneration process of generating tracking image data from the capturedimage data, the terminal-side controller performs a tracking calculationprocess of acquiring the target information on the basis of the trackingimage data, and transmits the target information to the camera-sidecontroller via the terminal-side communication unit and the camera-sidecommunication unit, the tracking image data is data of an imageincluding a high-resolution region with relatively higher resolution anda low-resolution region with relatively lower resolution, in thetracking image generation process, basic tracking information includinginformation on at least one of a size or a speed of the tracking targetis acquired, the high-resolution region and the low-resolution regionare determined on the basis of the basic tracking information, and thetracking image data is generated, and the tracking image generationprocess and the tracking calculation process are performed in parallel.

According to this aspect, since the tracking image data is generated onthe basis of information on at least one of the size or the speed of thetracking target, and the tracking image generation process and thetracking calculation process are performed in parallel, it is possibleto effectively speed up a process of tracking-imaging the trackingtarget while ensuring good tracking accuracy.

Another aspect of the present invention relates to a camera deviceconnectable to a terminal device including a terminal-side communicationunit, a terminal-side controller that is connected to the terminal-sidecommunication unit, and a user interface that is connected to theterminal-side controller, the camera device comprising: an imaging unitthat acquires the captured image data; an imaging direction adjustmentunit that adjusts an imaging direction of the imaging unit; and acamera-side controller that controls the imaging direction adjustmentunit on the basis of target information indicating a position of thetracking target, the camera-side controller performing a tracking imagegeneration process of generating tracking image data from the capturedimage data and a tracking calculation process of acquiring the targetinformation on the basis of the tracking image data in parallel, inwhich the tracking image data is data of an image including ahigh-resolution region with relatively higher resolution and alow-resolution region with relatively lower resolution, and in thetracking image generation process, basic tracking information includinginformation on at least one of a size or a speed of the tracking targetis acquired, the high-resolution region and the low-resolution regionare determined on the basis of the basic tracking information, and thetracking image data is generated.

According to this aspect, since the tracking image data is generated onthe basis of information on at least one of the size or the speed of thetracking target, and the tracking image generation process and thetracking calculation process are performed in parallel, it is possibleto effectively speed up a process of tracking-imaging the trackingtarget while ensuring good tracking accuracy.

Still another aspect of the present invention relates to a trackingimaging method of a tracking system comprising a camera device includingan imaging unit that acquires captured image data, an imaging directionadjustment unit that adjusts an imaging direction of the imaging unit, acamera-side controller that controls the imaging direction adjustmentunit on the basis of target information indicating a position of atracking target, and a camera-side communication unit that is connectedto the camera-side controller, and a terminal device including aterminal-side communication unit that is able to communicate with thecamera-side communication unit, a terminal-side controller that isconnected to the terminal-side communication unit, and a user interfacethat is connected to the terminal-side controller, the tracking imagingmethod comprising: a step of performing a tracking image generationprocess of generating tracking image data from the captured image databy at least one of the camera-side controller or the terminal-sidecontroller; and a step of performing a tracking calculation process ofacquiring the target information on the basis of the tracking image databy at least one of the camera-side controller or the terminal-sidecontroller, in which the tracking image data is data of an imageincluding a high-resolution region with relatively higher resolution anda low-resolution region with relatively lower resolution, in thetracking image generation process, basic tracking information includinginformation on at least one of a size or a speed of the tracking targetis acquired, the high-resolution region and the low-resolution regionare determined on the basis of the basic tracking information, and thetracking image data is generated, and the tracking image generationprocess and the tracking calculation process are performed in parallel.

Still another aspect of the present invention relates to a program forcontrolling a tracking system comprising a camera device including animaging unit that acquires captured image data, an imaging directionadjustment unit that adjusts an imaging direction of the imaging unit, acamera-side controller that controls the imaging direction adjustmentunit on the basis of target information indicating a position of atracking target, and a camera-side communication unit that is connectedto the camera-side controller, and a terminal device including aterminal-side communication unit that is able to communicate with thecamera-side communication unit, a terminal-side controller that isconnected to the terminal-side communication unit, and a user interfacethat is connected to the terminal-side controller, the program causing acomputer to execute: a step of performing a tracking image generationprocess of generating tracking image data from the captured image databy at least one of the camera-side controller or the terminal-sidecontroller; and a step of performing a tracking calculation process ofacquiring the target information on the basis of the tracking image databy at least one of the camera-side controller or the terminal-sidecontroller, in which the tracking image data is data of an imageincluding a high-resolution region with relatively higher resolution anda low-resolution region with relatively lower resolution, in thetracking image generation process, basic tracking information includinginformation on at least one of a size or a speed of the tracking targetis acquired, the high-resolution region and the low-resolution regionare determined on the basis of the basic tracking information, and thetracking image data is generated, and the tracking image generationprocess and the tracking calculation process are performed in parallel.

According to the present invention, since the tracking image data isgenerated on the basis of information on at least one of the size or thespeed of the tracking target, and the tracking image generation processand the tracking calculation process are performed in parallel, it ispossible to effectively speed up a process of tracking-imaging thetracking target while ensuring good tracking accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an external perspective view illustrating an exampleapplicable to a tracking system of the present invention.

FIG. 2 is a block diagram illustrating an example of function related totracking imaging in a functional configuration of the tracking system.

FIG. 3 is a conceptual diagram of a tracking image generation process.

FIG. 4 is a conceptual diagram of a tracking calculation process.

FIG. 5A is a conceptual diagram illustrating an example of capturedimage data, and FIG. 5B is a conceptual diagram illustrating an exampleof tracking image data.

FIG. 6 is a conceptual diagram illustrating an example of a trackingimage generation process.

FIG. 7 is a diagram illustrating an image example of tracking image dataof which resolution is adjusted on the basis of basic trackinginformation.

FIG. 8 is a diagram illustrating an example in which the tracking imagegeneration process and the tracking calculation process are performed inseries.

FIG. 9 is a diagram illustrating an example in which the tracking imagegeneration process and the tracking calculation process are performed inparallel.

FIG. 10 is a diagram illustrating an example of a tracking process flowaccording to a first embodiment.

FIG. 11 is a diagram illustrating an example of a tracking process flowaccording to a second embodiment.

FIG. 12 is a diagram illustrating an example of a tracking process flowaccording to a third embodiment.

FIG. 13 is a diagram illustrating an example of a tracking process flowaccording to a fourth embodiment.

FIG. 14 is a conceptual diagram illustrating an example of a pre-searchprocess.

FIG. 15 is a diagram illustrating an example of a process flow from apre-search process to a resolution determination process according to afifth embodiment.

FIG. 16 is a diagram illustrating an example of a process flow from apre-search process to a resolution determination process according to asixth embodiment.

FIG. 17 is a diagram illustrating an example of a process flow from apre-search process to a resolution determination process according to aseventh embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 is an external perspective view illustrating an example of atracking system 10 to which the present invention is applicable.

The tracking system 10 of this example includes a terminal device 11having a form of a smart phone, and a camera device 20 that has a formof a pan and tilt camera system and is connectable to the terminaldevice 11.

In the camera device 20, a holding unit 25 including a gear 26 and animaging optical system 21 attached to the holding unit 25 are fixedlymounted on a pedestal 23 provided on a device main body 22. The pedestal23 is provided to be rotatable about an axis Z in a vertical directionof the device main body 22, and a panning motion around the axis Z inthe vertical direction is performed by a pan driving unit (notillustrated). The gear 26 is provided coaxially with a horizontal axisX. When driving force is transmitted from a tilt driving unit (notillustrated) via the gear 26, the imaging optical system 21 is movedrotationally in a vertical direction and a tilting operation isperformed. The imaging optical system 21, the holding unit 25 (the gear26), and the pedestal 23 are covered with a dustproof and drip-proofdome cover 24. In FIG. 1, an optical axis of the imaging optical system21 is denoted with a reference sign “L”.

Meanwhile, a display 13, an operation panel 14, an operation button 16,a speaker 17, a microphone 18, and a terminal camera 19 are accommodatedin a housing 12 of the terminal device 11 of this example. The display13 functions as a display unit that displays an image or various typesof information transmitted from the camera device 20, and constitutes atouch panel 15 together with the operation panel 14. The operation panel14 and the operation button 16 function as an instruction reception unitthat receives an instruction from the user, and the user can operate theterminal device 11 and the camera device 20 connected to the terminaldevice 11 via the operation panel 14 (the touch panel 15) and theoperation button 16. The speaker 17 and the microphone 18 function as acalling unit, and the user can talk with a user of another phone throughthe speaker 17 and microphone 18. The terminal camera 19 may performimaging according to an instruction from the user via the operationbutton 16 or the operation panel 14 (the touch panel 15).

The user can control an operation of the camera device 20 using theterminal device 11 described above. For example, the captured image canbe transmitted from the camera device 20 to the terminal device 11 anddisplayed on the display 13, and an imaging direction of the cameradevice 20 (a direction of an optical axis L of the imaging opticalsystem 21) can be changed. In particular, the tracking system 10 of thisexample can execute tracking imaging for continuously performing imagingwhile tracking a specific target, and the user can control the trackingimaging using the terminal device 11. Hereinafter, tracking imaging willbe described.

FIG. 2 is a block diagram illustrating an example of a function relatedto tracking imaging in a functional configuration of the tracking system10.

The camera device 20 includes an imaging unit 30, an imaging directionadjustment unit 32, a camera-side communication unit 36, and acamera-side controller 34 that generally controls the imaging unit 30,the imaging direction adjustment unit 32, and the camera-sidecommunication unit 36.

The imaging unit 30 includes the imaging optical system 21 (see FIG. 1)and an imaging element (not illustrated), and acquires captured imagedata. In particular, since the camera device 20 of this example is atracking camera that continuously performs imaging of a subject, theimaging unit 30 can regularly perform imaging under the control of thecamera-side controller 34 and outputs acquired captured image data.

The imaging direction adjustment unit 32 of this example is configuredas a pan and tilt mechanism including a pan driving unit and a tiltdriving unit (not illustrated), in addition to the pedestal 23, theholding unit 25, and the gear 26 illustrated in FIG. 1, and can adjustan imaging direction of the imaging unit 30 (that is, a direction of theoptical axis L of the imaging optical system 21).

The camera-side controller 34 controls the imaging direction adjustmentunit 32 on the basis of the target information indicating the positionof the tracking target. In particular, the camera-side controller 34 ofthis example controls the imaging direction adjustment unit 32 to adjustthe imaging direction so that the tracking target is arranged at aspecific position or within a specific range in the captured image. The“specific position” or the “specific range” described herein is notparticularly limited, but it is preferable for the specific position orthe specific range to be set to a position or a range corresponding to acentral portion of the captured image (that is, a position or a rangecorresponding to the optical axis L) from the viewpoint of ensuring goodvisibility of the tracking target. The target information is calculatedthrough a tracking calculation process to be described below and isprovided to the camera-side controller 34.

Therefore, for example, in a case where the tracking target moves in ahorizontal direction, the imaging direction adjustment unit 32pan-drives the imaging unit 30 under the control of the camera-sidecontroller 34 so that the imaging direction moves in the horizontaldirection according to the movement of the tracking target. Similarly,in a case where the tracking target moves in a vertical direction, theimaging direction adjustment unit 32 tilt-drives the imaging unit 30under the control of the camera-side controller 34 so that the imagingdirection moves in the vertical direction according to the movement ofthe tracking target. By continuously performing imaging of the trackingtarget while adjusting the imaging direction in this way, trackingimaging is performed.

The camera-side communication unit 36 is connected to the camera-sidecontroller 34, performs communication with the terminal device 11(particularly, a terminal-side communication unit 44 to be describedbelow) (see an arrow “C” in FIG. 2), performs transmission of data thatis sent from the camera device 20 to the terminal device 11, andperforms reception of data that is sent from the terminal device 11 tothe camera device 20. That is, the camera-side communication unit 36transmits data sent from the camera-side controller 34 to the terminaldevice 11 (the terminal-side communication unit 44), and receives datasent from the terminal device 11 (the terminal-side communication unit44) and sends the data to the camera-side controller 34.

The data transmitted and received between the camera device 20 (thecamera-side communication unit 36) and the terminal device 11 (theterminal-side communication unit 44) is not particularly limited. Forexample, basic tracking information including information on at leastone of captured image data acquired by the imaging unit 30, trackingimage data generated through a tracking image generation process to bedescribed below, the above-described target information, a size of thetracking target or a speed of the tracking target, and various commandscan be transmitted and received between the camera device 20 and theterminal device 11, if necessary.

Meanwhile, the terminal device 11 which can be connected to the cameradevice 20 includes a user interface 42, the terminal-side communicationunit 44, and a terminal-side controller 40 that generally controls theuser interface 42 and the terminal-side communication unit 44.

The terminal-side communication unit 44 can communicate with thecamera-side communication unit 36, receives data sent from thecamera-side communication unit 36, sends the data to the terminal-sidecontroller 40, and transmits data sent from the terminal-side controller40 to the camera-side communication unit 36. A scheme of communicationbetween the terminal-side communication unit 44 and the camera-sidecommunication unit 36 is not particularly limited, and may be a wiredconnection scheme or may be a wireless connection scheme. Further, thecommunication scheme may be a communication scheme based on a wirelesslocal area network (LAN) conforming to the IEEE 802.11a/b/g/n standarddefined by The Institute of Electrical and Electronics Engineers, Inc.(IEEE), or the like, or may be a communication scheme based on NearField Wireless confirming to a Bluetooth (registered trademark) standardor the like. Further, the terminal-side communication unit 44 mayperform transmission and reception of data to and from a base station(not illustrated) on the basis of a 3rd generation mobile communicationsystem conforming to the IMT-2000 standard or a 4th generation mobilecommunication system conforming to the IMT-Advance standard.

The terminal-side controller 40 is connected to the terminal-sidecommunication unit 44 and the user interface 42, performs variouscontrols on the basis of data such as commands sent from the userinterface 42, and controls the terminal-side communication unit 44 tosend the data to the camera-side communication unit 36, as necessary.

The user interface 42 constitutes an interface to the user of theterminal device 11. Thus, for example, the operation button 16 and thetouch panel 15 (the operation panel 14) that receive an instruction fromthe user, the display 13 that can display an image or the like, thespeaker 17 that provides sound to the user, and the microphone 18 thatacquires sound may be included in the user interface 42.

In the tracking system 10 having the functional configuration describedabove, at least one of the camera-side controller 34 or theterminal-side controller 40 performs the tracking image generationprocess of generating the tracking image data from the captured imagedata. Further, at least one of the camera-side controller 34 or theterminal-side controller 40 performs the tracking calculation process ofacquiring target information on the basis of the tracking image data.

FIG. 3 is a conceptual diagram of a tracking image generation processP1. FIG. 4 is a conceptual diagram of a tracking calculation process P2.In the tracking image generation process P1, tracking image data D2 isgenerated from the captured image data D1, and in the trackingcalculation process P2, target information D3 is acquired from thetracking image data D2. A specific algorithm for the tracking imagegeneration process P1 and the tracking calculation process P2 is notparticularly limited, and can generate the tracking image data D2 andacquire the target information D3 according to an arbitrary scheme. Forexample, in the tracking image generation process P1, the tracking imagedata D2 suitable for acquisition of the target information D3 can begenerated by changing resolution of a part of the captured image dataD1.

FIG. 5 is a conceptual diagram illustrating an example of the capturedimage data D1 and the tracking image data D2, a portion (a) of FIG. 5illustrates the captured image data D1, and a portion (b) of FIG. 5illustrates the tracking image data D2. The captured image data D1 ofthis example is data of an image having uniform resolution over theentire image, whereas the tracking image data D2 is data of an imageincluding the high-resolution region RH of which the resolution isrelatively high and the high-resolution region RL of which theresolution is relatively low. That is, the captured image data D1 isconstituted by a set of first pixels PS having a certain size, whereasthe tracking image data D2 includes the high-resolution region RHconstituted by a set of second pixels PH having a relatively small pixelsize, and the low-resolution region RL constituted by a set of thirdpixels PL having a relatively large pixel size.

A specific scheme of setting the high-resolution region RH and thelow-resolution region RL is not particularly limited. For example, thesecond pixel PH of the high-resolution region RH may be constituted bythe same pixel as the first pixel PS of the captured image data D1, andthe third pixel PL of the low-resolution region RL may be constituted by“some pixels selected from among the first pixels PS of the capturedimage data D1” or “representative pixels derived from a plurality ofadjacent pixels”. In this example, a region in which the number ofpixels included in a unit area of the image is relatively large isreferred to as the high-resolution region RH, and a region in which thenumber of pixels included in the unit area of the image is relativelysmall is referred to as the low-resolution region RL.

A position and a range of the high-resolution region RH and thelow-resolution region RL in the tracking image data D2 are notparticularly limited, but in this example, the position and the rangemay be determined on the basis of the basic tracking information in thetracking image generation process P1.

FIG. 6 is a conceptual diagram illustrating an example of the trackingimage generation process P1. The tracking image generation process P1 ofthis example includes a resolution determination process P3 and aresolution adjustment process P4. In the resolution determinationprocess P3, resolution information D5 for determining thehigh-resolution region RH and the low-resolution region RL in thetracking image data D2 is obtained on the basis of basic trackinginformation D4 including the information on at least one of the size orthe speed of the tracking target. In the resolution adjustment processP4, the tracking image data D2 is generated on the basis of theresolution information D5 and the captured image data D1. Thus, in thetracking image generation process P1 of this example, the basic trackinginformation D4 including the information on at least one of the size orthe speed of the tracking target is acquired, the high-resolution regionRH and the low-resolution region RL are determined on the basis of thebasic tracking information D4, and the tracking image data D2 isgenerated.

A scheme of acquiring the basic tracking information D4 and the capturedimage data D1 in the tracking image generation process P1 is notparticularly limited. For example, the basic tracking information D4 andthe captured image data D1 held in a memory (not illustrated) may beappropriately read, and the basic tracking information D4 and thecaptured image data D1 output from each unit of the terminal device 11and/or the camera device 20 may be appropriately acquired. For example,in the camera-side controller 34 and/or the terminal-side controller 40,the size and the speed of the tracking target may be acquired on thebasis of the captured image data D1, and information on the acquiredsize and the acquired speed of the tracking target may be used as thebasic tracking information D4.

Further, the basic tracking information D4 may include information otherthan the size and the speed of the tracking target and, for example,information indicating processing ability of the camera device 20 (thecamera-side controller 34 and the camera-side communication unit 36) andthe terminal device 11 (the terminal-side communication unit 44 and theterminal-side controller 40) that are actually used may be included inthe basic tracking information D4.

Further, a scheme of designating the tracking target is not particularlylimited. For example, when a live view image is generated from thecaptured image data D1 by the camera device 20 (the camera-sidecontroller 34) or the terminal device 11 (the terminal-side controller40), and the live view image is displayed in the display 13 (the userinterface 42) of terminal device 11, the user can designate the positionof the tracking target through the touch panel 15. In this case,position designation information of the tracking target may be sent fromthe touch panel 15 (the user interface 42) to the terminal-sidecontroller 40 and/or the camera-side controller 34, and theterminal-side controller 40 and/or the camera-side controller 34 mayspecify a specific tracking target on the basis of the positiondesignation information of the tracking target. Further, the camera-sidecontroller 34 and/or the terminal-side controller 40 may automaticallyspecify a specific subject such as a face on the basis of the capturedimage data D1 and set the specified specific subject as the trackingtarget. Further, by performing a moving object detection process ofspecifying a subject with motion using the camera-side controller 34and/or the terminal-side controller 40, for example, a subject newlyintruded into the imaging range of the imaging unit 30 may be set as thetracking target.

FIG. 7 is a diagram illustrating an image example of the tracking imagedata D2 of which the resolution has been adjusted on the basis of thebasic tracking information D4. In this example, a face image of a personis set as the tracking target T, the basic tracking information D4includes position and speed information of the tracking target T, arange near the tracking target T in the tracking image data D2 is set tothe high-resolution region RH, and the other range is set to thelow-resolution region RL. The high-resolution region RH and thelow-resolution region RL of the tracking image data D2 are set in thisway, making it possible to set a range in which an existence probabilityof the tracking target T is estimated to be relatively high and enhancedtracking accuracy is required, to the high-resolution region RH.Further, it is possible to set a range in which an existence probabilityof the tracking target T is estimated to be relatively low to thelow-resolution region RL, and it is possible to effectively reduce aprocessing load and a processing time.

A range of the high-resolution region RH determined on the basis of thebasic tracking information D4 is not particularly limited. For example,the range may be set according to only the position of the trackingtarget T, may be set variably according to the speed of the trackingtarget T, or may be variably set according to elements other than theposition and the speed of the tracking target T. For example, in a casewhere the tracking target T is present in a central portion of thetracking image data D2 (and, the captured image data D1) and a casewhere the tracking target T is present in an end portion of the trackingimage data D2, the range of the high-resolution region RH may bedifferent. Further, in a case where the speed of the tracking target Tis relatively high, the range of the high-resolution region RH may bewider than in a case where the speed of the tracking target T isrelatively slow. Further, the range of the high-resolution region RH maybe determined variably according to, for example, an acceleration of thetracking target T, the movement direction of the tracking target T,and/or another feature of the tracking target T, in addition to theposition and/or the speed of the tracking target T. Further, in a casewhere the basic tracking information D4 includes information on theprocessing capability of the camera device 20 and the terminal device11, the ranges of the high-resolution region RH and the low-resolutionregion RL may be determined, for example, according to the processingcapability of the camera device 20 and the terminal device 11.

Further, a shape of the high-resolution region RH is not limited to asquare shape and, for example, the high-resolution region RH in anotherpolygonal shape or a circular shape may be set. Further, the shape ofthe high-resolution region RH may be set according to the position, thespeed, and/or another element of the tracking target T.

Next, a specific embodiment of the tracking image generation process P1and the tracking calculation process P2 will be described. The trackingsystem 10 of this example performs the tracking image generation processP1 and the tracking calculation process P2 described above in parallel.

FIG. 8 is a diagram illustrating an example in which the tracking imagegeneration process P1 and the tracking calculation process P2 areperformed in series. In FIG. 8, reference signs “P1-1”, “P1-2”, and“P1-3” indicate the tracking image generation processes P1 regarding thefirst timing, the second timing, and the third timing, respectively, andreference signs “P2-1”, “P2-2”, and “P2-3” indicate the trackingcalculation processes P2 regarding the first timing, the second timing,and the third timing, respectively. The second timing refers to a timingafter the first timing, and the third timing refers to a timing afterthe second timing.

In the tracking image generation process P1-1 regarding the firsttiming, the tracking image data D2 is generated from the captured imagedata D1 acquired at the first timing. In the tracking image generationprocess P1-2 regarding the second timing, the tracking image data D2 isgenerated from the captured image data D1 acquired at the second timing.In the tracking image generation process P1-3 regarding the thirdtiming, the tracking image data D2 is generated from the captured imagedata D1 acquired at the third timing. Further, in the trackingcalculation process P2-1 regarding the first timing, the targetinformation D3 indicating the position of the tracking target at thefirst timing is acquired on the basis of the tracking image data D2generated through the tracking image generation process P1-1 regardingthe first timing. In the tracking calculation process P2-2 regarding thesecond timing, the target information D3 indicating the position of thetracking target at the second timing is acquired on the basis of thetracking image data D2 generated through the tracking image generationprocess P1-2 regarding the second timing. In the tracking calculationprocess P2-3 regarding the third timing, the target information D3indicating the position of the tracking target at the third timing isacquired on the basis of the tracking image data D2 generated throughthe tracking image generation process P1-3 regarding the third timing.

In a case where the tracking image generation process P1 and thetracking calculation process P2 are performed in series in this way,time equal to or longer than a sum of a processing time of the trackingimage generation process P1 and a processing time of the trackingcalculation process P2 is required as a total processing time. That is,in the example illustrated in FIG. 8, a sum (t1 to t7) of time (t1 tot2) required for the tracking image generation process P1-1 regardingthe first timing, time (t2 to t3) required for the tracking calculationprocess P2-1 regarding the first timing, time (t3 to t4) required forthe tracking image generation process P1-2 regarding the second timing,time (t4 to t5) required for the tracking calculation process P2-2regarding the second timing, time (t5 to t6) required for the trackingimage generation process P1-3 regarding the third second timing, andtime (t6 to t7) required for the tracking calculation process P2-3regarding the third timing is necessary to acquire the targetinformation D3 at the first to third timings.

On the other hand, in the tracking system 10 of this example, thetracking image generation process P1 and the tracking calculationprocess P2 are performed in parallel, making it possible to greatlyreduce the total processing time.

FIG. 9 is a diagram illustrating an example in which the tracking imagegeneration process P1 and the tracking calculation process P2 areperformed in parallel. In the example illustrated in FIG. 9, thetracking calculation process P2-1 regarding the first timing and thetracking image generation process P1-2 regarding the second timing areperformed in parallel, the tracking calculation process P2-2 regardingthe second timing and the tracking image generation process P1-3regarding the third timing are performed in parallel, and the trackingcalculation process P2-3 regarding the third timing and the trackingimage generation process P1-4 regarding the fourth timing are performedin parallel.

In an example illustrated in FIG. 9, a time of period “t1 to t2” isrepresented as a period of time required for the tracking imagegeneration process P1-1 regarding the first timing, a time of period “t2to t3” is represented as a period of time required for the trackingcalculation process P2-1 regarding the first timing and the trackingimage generation process P1-2 regarding the second timing, a time ofperiod “t3 to t4” is represented as a period of time required for thetracking calculation process P2-2 regarding the second timing and thetracking image generation process P1-3 regarding the third timing, and atime of period “t4 to t5” is represented as a period of time requiredfor the tracking calculation process P2-3 regarding the third timing andthe tracking image generation process P1-4 regarding the fourth timing.

Particularly, in the tracking system 10 of this example, the imagingdirection is adjusted so that the tracking target is arranged at aspecific position or in a specific range in the captured image.Therefore, the high-resolution region RH and the low-resolution regionRL of the tracking image data D2 generated through the tracking imagegeneration process P1 can be determined with reference to the specificposition or the specific range, and the tracking image generationprocess P1 and the tracking calculation process P2 can be performed inparallel as illustrated in FIG. 9.

For example, in a case where the tracking target is not placed at aspecific position or in a specific range in the image, a position in theimage to which the tracking target moves cannot be known. Accordingly,the high-resolution region RH cannot be specified until the trackingcalculation process P2 has completely ended, and it is necessary for thetracking image generation process P1 and the tracking calculationprocess P2 to be performed in series. Meanwhile, when the imagingdirection is adjusted by the imaging direction adjustment unit 32 suchas a pan and tilt mechanism as in this example and the tracking targetis placed at a specific position or in specific range of the capturedimage, the position of the tracking target is basically maintained atthe specific position or in the specific range through tracking imaging.Accordingly, in a case where the tracking target is placed at thespecific position or in the specific range in the captured image, thetracking image data D2 can be generated from the captured image data D1on the basis of the specific position or the specific range in thecaptured image in which the tracking target is expected to be placedeven when an exact position of the tracking target at each timing is notspecified. In the case where the imaging direction is controlled so thatthe tracking target is placed at the specific position or in thespecific range in the captured image at the time of tracking-imaging inthis way, the tracking image generation process P1 regarding the nexttiming (frame) can be performed without waiting for completion of thetracking calculation process P2 regarding an immediately previous timing(frame). Accordingly, the tracking image generation process P1 and thetracking calculation process P2 can be performed in parallel.

At least a portion of the tracking calculation process P2-1 of acquiringthe target information D3 indicating the position of the tracking targetat the first timing on the basis of the tracking image data D2 generatedfrom the captured image data D1 acquired at the first timing isperformed simultaneously with at least a portion of the tracking imagegeneration process P1-2 of generating the tracking image data D2 that isused to acquire the target information D3 indicating the position of thetracking target at the second timing after the first timing, from thecaptured image data D1 acquired at the second timing. Thus, a parallelprocess may be performed, and the present invention is not limited tothe example illustrated in FIG. 9.

That is, “at least a portion of the tracking image generation processPI” and “at least a portion of the tracking calculation process P2”regarding a different timing from that of the tracking image generationprocess P1 may temporally overlap in time. Thus, for example, while thewhole of one of the tracking image generation process P1 and thetracking calculation process P2 of which the timings are different isbeing performed, the whole of the other may be performed.

By performing the tracking image generation process P1 and the trackingcalculation process P2 in parallel in this way, it is possible togreatly reduce time required for the entire tracking process, ascompared with a case where the tracking image generation process P1 andthe tracking calculation process P2 are performed in series (see FIG.8).

The tracking image generation process P1 and the tracking calculationprocess P2 may be performed in any one of the camera-side controller 34and the terminal-side controller 40, or may be performed in both.

Hereinafter, a representative embodiment of the tracking system 10 willbe described.

First Embodiment

In this embodiment, both the tracking image generation process P1 andthe tracking calculation process P2 are performed in the terminal device11 (the terminal-side controller 40). That is, the terminal-sidecontroller 40 performs the tracking image generation process P1 ofgenerating the tracking image data D2 from the captured image data D1.Further, the terminal-side controller 40 performs the trackingcalculation process P2 of acquiring the target information D3 on thebasis of the tracking image data D2, and transmits the targetinformation D3 to the camera-side controller 34 via the terminal-sidecommunication unit 44 and the camera-side communication unit 36.

FIG. 10 is a diagram illustrating an example of a tracking process flowaccording to the first embodiment.

The captured image data D1 acquired by the imaging unit 30 of the cameradevice 20 at the time of tracking-imaging is transmitted from thecamera-side communication unit 36 under the control of the camera-sidecontroller 34 (S11 in FIG. 10), is received by the terminal-sidecommunication unit 44 of the terminal device 11 (S12), and is acquiredby the terminal-side controller 40.

The tracking image generation process P1 of generating the trackingimage data D2 from the captured image data D1 is performed by theterminal-side controller 40 (S13), and then, the tracking calculationprocess P2 of acquiring the target information D3 on the basis of thetracking image data D2 is performed (S14).

The target information D3 acquired by the terminal-side controller 40 istransmitted from the terminal-side communication unit 44 (S15), isreceived by the camera-side communication unit 36 (S16), and is acquiredby the camera-side controller 34.

The camera-side controller 34 controls the imaging direction adjustmentunit 32 on the basis of the target information D3 that is acquired inthis way to adjust an imaging direction of the imaging unit 30 (adirection of the optical axis L of the imaging optical system 21) (S17),making it possible to perform imaging while tracking the trackingtarget. In this embodiment, the imaging direction is adjusted so thatthe tracking target is arranged at a specific position or in a specificrange in the captured image.

The above-described processes (S11 to S17) illustrated in FIG. 10 areperformed for the respective imaging timings (see the “first timing” to“fourth timing” described above), steps S11, S16, and S17 describedabove are performed by the camera device 20, and steps S12 to S15 areperformed by the terminal device 11. The tracking image generationprocess P1 (S13) and the tracking calculation process P2 (S14) areperformed in parallel, and for example, at least a portion of thetracking calculation process P2-1 (S14) regarding the first timing andat least a portion of the tracking image generation process P1-2 (S13)regarding the second timing are simultaneously performed (see FIG. 9).

Accordingly, it is possible to greatly shorten a processing time of theentire tracking process and supped up the tracking process and, as aresult, it is possible to appropriately track a fast moving trackingtarget and ensure good tracking accuracy.

Second Embodiment

One of the tracking image generation process P1 and the trackingcalculation process P2 described above may be performed in one of thecamera-side controller 34 and the terminal-side controller 40, and theother of the tracking image generation process P1 and the trackingcalculation process P2 may be performed in the other of the camera-sidecontroller 34 and the terminal-side controller 40.

In this embodiment, a case where the tracking image generation processP1 is performed in the camera-side controller 34, and the trackingcalculation process P2 is performed in the terminal-side controller 40will be described by way of example. In this embodiment, detaileddescription of a configuration that is the same as or similar to that inthe first embodiment described above will be omitted.

FIG. 11 is a diagram illustrating an example of a tracking process flowaccording to the second embodiment.

At the time of tracking-imaging, the camera-side controller 34 of thecamera device 20 performs the tracking image generation process P1 ofgenerating the tracking image data D2 from the captured image data D1acquired by the imaging unit 30 (S21 in FIG. 11). The tracking imagedata D2 generated through the tracking image generation process P1 istransmitted from the camera-side communication unit 36 under the controlof the camera-side controller 34 (S22), received by the terminal-sidecommunication unit 44 of the terminal device 11 (S23), and is acquiredby the terminal-side controller 40. Other data such as the capturedimage data D1 together with the tracking image data D2 may be sent fromthe camera-side communication unit 36 to the terminal-side communicationunit 44.

The tracking calculation process P2 of acquiring the target informationD3 is performed on the basis of the tracking image data D2 by theterminal-side controller 40 (S24). The acquired target information D3 istransmitted from the terminal-side communication unit 44 (S25), isreceived by the camera-side communication unit 36 (S26), and is acquiredby the camera-side controller 34.

The camera-side controller 34 controls the imaging direction adjustmentunit 32 on the basis of the target information D3 that is acquired inthis way, and adjusts the imaging direction of the imaging unit 30 sothat the tracking target is arranged at a specific position or in aspecific range in the captured image (S27).

The above-described processes (S21 to S27) illustrated in FIG. 11 areperformed for the respective imaging timings (see the “first timing” to“fourth timing” described above), similar to the first embodimentdescribed above. The tracking image generation process P1 (S21) in thecamera-side controller 34 and the tracking calculation process P2 (S24)in the terminal-side controller 40 are performed in parallel, and forexample, at least a portion of the tracking calculation process P2-1(S24) regarding the first timing and at least a portion of the trackingimage generation process P1-2 (S21) regarding the second timing aresimultaneously performed (see FIG. 9).

Third Embodiment

In this embodiment, an example in which the tracking image generationprocess P1 and the tracking calculation process P2 are performed by thecamera-side controller 34 will be described. That is, the camera-sidecontroller 34 of this embodiment performs the tracking image generationprocess P1 of generating the tracking image data D2 from the capturedimage data D1 and the tracking calculation process P2 of acquiring thetarget information D3 on the basis of the tracking image data D2 inparallel.

In this embodiment, detailed description of a configuration that is thesame as or similar to that in the first embodiment described above willbe omitted.

FIG. 12 is a diagram illustrating an example of a tracking process flowaccording to the third embodiment.

At the time of tracking-imaging, the camera-side controller 34 of thecamera device 20 performs the tracking image generation process P1 ofgenerating the tracking image data D2 from the captured image data D1acquired by the imaging unit 30 (S31 in FIG. 12). Further, thecamera-side controller 34 performs the tracking calculation process P2of acquiring the target information D3 on the basis of the trackingimage data D2 (S32). The camera-side controller 34 controls the imagingdirection adjustment unit 32 on the basis of the target information D3that is acquired in this way, and adjusts the imaging direction of theimaging unit 30 so that the tracking target is arranged at a specificposition or in a specific range in the captured image (S33).

The above-described processes (S31 to S33) illustrated in FIG. 12 areperformed for the respective imaging timings (see the “first timing” to“fourth timing” described above), as in the first embodiment describedabove. The tracking image generation process P1 (S31) and the trackingcalculation process P2 (S32) in the camera-side controller 34 areperformed in parallel, and for example, at least a portion of thetracking calculation process P2-1 (S32) regarding the first timing andat least a portion of the tracking image generation process P1-2 (S31)regarding the second timing are simultaneously performed (see FIG. 9).

Fourth Embodiment

The tracking calculation process P2 may be performed by both of thecamera-side controller 34 and the terminal-side controller 40. In thiscase, it is preferable that a controller having a relatively higherprocessing capability between the camera-side controller 34 and theterminal-side controller 40 performs the tracking calculation process P2of the high-resolution region RH, and a controller having a relativelylower processing capability between the camera-side controller 34 andthe terminal-side controller 40 performs the tracking calculationprocess P2 of the low-resolution region RL.

In this embodiment, an example of a case in which the terminal-sidecontroller 40 has a higher processing capability than the camera-sidecontroller 34, the terminal-side controller 40 performs the trackingcalculation process P2 of the high-resolution region RH, and thecamera-side controller 34 performs the tracking calculation process P2of the low-resolution region RL will be described. In this embodiment,detailed description of a configuration that is the same as or similarto that in the first embodiment described above will be omitted.

FIG. 13 is a diagram illustrating an example of a tracking process flowaccording to the fourth embodiment.

At the time of tracking-imaging, the camera-side controller 34 of thecamera device 20 performs the tracking image generation process P1 ofgenerating the tracking image data D2 from the captured image data D1acquired by the imaging unit 30 (S41 in FIG. 13). The tracking imagedata D2 generated through the tracking image generation process P1 istransmitted from the camera-side communication unit 36 under the controlof the camera-side controller 34 (S42), is received by the terminal-sidecommunication unit 44 of the terminal device 11 (S43), and is acquiredby the terminal-side controller 40.

The tracking calculation process P2 is performed regarding thehigh-resolution region RH of the tracking image data D2 by theterminal-side controller 40 (S44). A result of the tracking calculationprocess P2 regarding the high-resolution region RH is transmitted fromthe terminal-side communication unit 44 (S45), and is acquired by thecamera-side controller 34 through the camera-side communication unit 36(S47).

Meanwhile, in the camera-side controller 34, the tracking imagegeneration process P1 (S41) is performed, the tracking calculationprocess P2 is performed regarding the low-resolution region RL of thetracking image data D2 (S46), and then, a result of the trackingcalculation process P2 regarding the low-resolution region RL isacquired (S47).

The camera-side controller 34 acquires the target information D3 on thebasis of the “result of the tracking calculation process P2 regardingthe high-resolution region RH of the tracking image data D2” and the“result of the tracking calculation process P2 regarding thelow-resolution region RL of the tracking image data D2” (S48). Thecamera-side controller 34 controls the imaging direction adjustment unit32 on the basis of the target information D3 that is acquired in thisway, and adjusts the imaging direction of the imaging unit 30 so thatthe tracking target is arranged at a specific position or in a specificrange in the captured image (S49).

The above-described processes (S41 to S49) illustrated in FIG. 13 areperformed for the respective imaging timings (see the “first timing” to“fourth timing” described above), similar to the first embodimentdescribed above. The “tracking image generation process P1 (S41) in thecamera-side controller 34” and the “tracking calculation process P2 (S44and S46) in the camera-side controller 34 and the terminal-sidecontroller 40” are performed in parallel. For example, at least aportion of the tracking calculation process P2-1 (S44 and S46) regardingthe first timing and at least a portion of the tracking image generationprocess P1-2 (S41) regarding the second timing are simultaneouslyperformed (see FIG. 9).

Fifth Embodiment

In this embodiment, prior to tracking imaging, a pre-search process ofacquiring a spatial frequency of the captured image data D1 of at leasta portion of the imaging range of the camera device 20 is performed, andresolution of the tracking image data D2 is determined according to aresult of the pre-search process.

Here, the term “imaging range of the camera device 20” can be determinedaccording to a variable range of the imaging direction (a direction ofthe optical axis L of the imaging optical system 21). For example, inthe tracking system 10 in which the pan and tilt mechanism is adapted asillustrated in FIG. 1, the “imaging range of the camera device 20” isdetermined according to a range in which a panning operation and atilting operation can be performed.

In this embodiment, detailed description of a configuration that is thesame as or similar to that in the first embodiment described above willbe omitted.

FIG. 14 is a conceptual diagram illustrating an example of a pre-searchprocess P5.

The pre-search process P5 of acquiring the spatial frequency of thecaptured image data D1 of at least a portion of the imaging range of thecamera device 20 is performed by at least one of the camera-sidecontroller 34 or the terminal-side controller 40. Specifically, imagingin which a target is an imaging environment which is the imaging rangeof the camera device 20 is performed in advance, the captured image dataD1 of the imaging environment is acquired, the captured image data D1 ofthe imaging environment is analyzed in the pre-search process P5. Thus,environmental frequency information D6 indicating the spatial frequencyof the imaging environment is acquired.

The captured image data D1 used in the pre-search process P5 and theenvironmental frequency information D6 acquired through the pre-searchprocess P5 may be ones regarding a portion of the imaging range of thecamera device 20 or may be ones regarding all of the imaging range.Therefore, from the point of view of a reduction in time required forthe pre-search process P5, for example, only a central part of theimaging range of the camera device 20 may be a target of the capturedimage data D1 and the environmental frequency information D6 in thepre-search process P5.

Further, the acquisition of the captured image data D1 of the imagingenvironment may be automatically performed or may be manually performed.For example, the camera device 20 (the camera-side controller 34) andthe terminal device 11 (the terminal-side controller 40) mayautomatically cooperate with each other and acquire the captured imagedata D1 of the imaging environment, or the captured image data D1 may beacquired for a specific range of the imaging environment that ismanually designated by the user operating the terminal device 11 (seeFIGS. 1 and 2).

The environmental frequency information D6 acquired through thepre-search process P5 is used in the above-described resolutiondetermination process P3 (see FIG. 6). In the resolution determinationprocess P3 (the tracking image generation process P1), the resolution ofthe tracking image data D2 is determined on the basis of the spatialfrequency of the imaging environment (the environmental frequencyinformation D6) acquired through the pre-search process P5. In theresolution determination process P3 in the example illustrated in FIG.14, the resolution information D5 that determines the high-resolutionregion RH and the low-resolution region RL in the tracking image data D2is obtained on the basis of the basic tracking information D4 and theenvironmental frequency information D6.

In the resolution adjustment process P4 (tracking image generationprocess P1), the tracking image data D2 is generated on the basis of theresolution information D5 and the captured image data D1 determined inthis way. For example, in a case where the spatial frequency of theimaging environment acquired through the pre-search process P5 isrelatively low, the resolution of the tracking image data D2 is set tobe lower than that in a case where the spatial frequency of the imagingenvironment is relatively high. A scheme of setting the resolution ofthe tracking image data D2 is not particularly limited and, for example,in a case where the spatial frequency of the imaging environment isrelatively low, resolution of one or both of the “high-resolution regionRH of the tracking image data D2” and the “low-resolution region RL ofthe tracking image data D2” may be set to be lower than in a case wherethe spatial frequency of the imaging environments is relatively high.

Under an environment in which the imaging range of the camera device 20does not change as in a case where the camera device 20 is fixedlyinstalled, the pre-search process P5 may be performed only once beforethe tracking imaging. Further, the environmental frequency informationD6 acquired through the pre-search process P5 may be stored in a memory(not illustrated). In the resolution determination process P3 (thetracking image generation process P1), the environmental frequencyinformation D6 stored in the memory may be read and used to acquire theresolution information D5.

A case in which the pre-search process P5 is performed in thecamera-side controller 34 (the camera device 20), and the resolutiondetermination process P3 (the tracking image generation process P1) isperformed in the terminal-side controller 40 (the terminal device 11)regarding a series of processing flows from the pre-search process P5 tothe resolution determination process P3 will be described by way ofexample below.

FIG. 15 is a diagram illustrating an example of a process flow from thepre-search process P5 to the resolution determination process P3according to the fifth embodiment.

First, an imaging environment is imaged by the imaging unit 30 while theimaging direction is adjusted by the pan and tilt mechanism of thecamera device 20, and the captured image data D1 of the imagingenvironment is acquired by the camera-side controller 34 (S51 in FIG.15). In the camera-side controller 34, the pre-search process P5 isperformed on the basis of the captured image data D1 of the imagingenvironment and the environmental frequency information D6 is acquired(S52). This environmental frequency information D6 is transmitted fromthe camera-side communication unit 36 (S53), is received by theterminal-side communication unit 44 of the terminal device 11 (S54), andis acquired by the terminal-side controller 40.

When the tracking imaging is started, the terminal-side controller 40acquires the basic tracking information D4 at each imaging timing (see“the first timing” to the “fourth timing” described above) (S55), andperforms the resolution determination process P3 to acquire theresolution information D5 on the basis of the basic tracking informationD4 and the environmental frequency information D6 (S56). This acquiredresolution information D5 is supplied to the resolution adjustmentprocess P4 in a subsequent step.

Sixth Embodiment

In this embodiment, a case where the pre-search process P5 and theresolution determination process P3 is performed in the terminal-sidecontroller 40 (the terminal device 11) will be described. In thisembodiment, detailed description of a configuration that is the same asor similar to that in the fifth embodiment described above will beomitted.

FIG. 16 is a diagram illustrating an example of a process flow from thepre-search process P5 to the resolution determination process P3according to the sixth embodiment.

In this embodiment, first, an imaging environment is imaged by theimaging unit 30 while the imaging direction is adjusted by the pan andtilt mechanism of the camera device 20, and the captured image data D1of the imaging environment is acquired by the camera-side controller 34(S61 in FIG. 16). This captured image data D1 of the imaging environmentis transmitted from the camera-side communication unit 36 (S62), isreceived by the terminal-side communication unit 44 of the terminaldevice 11 (S63), and is acquired by the terminal-side controller 40.

In the terminal-side controller 40, the pre-search process P5 isperformed and the environmental frequency information D6 is acquired(S64).

When the tracking imaging is started, the terminal-side controller 40acquires the basic tracking information D4 at each imaging timing (see“the first timing” to the “fourth timing” described above) (S65), andperforms the resolution determination process P3 to acquire theresolution information D5 on the basis of the basic tracking informationD4 and the environmental frequency information D6 (S66).

Seventh Embodiment

In this embodiment, a case where the pre-search process P5 and theresolution determination process P3 are performed in the camera-sidecontroller 34 (the camera device 20) will be described. In thisembodiment, detailed description of a configuration that is the same asor similar to that in the fifth embodiment described above will beomitted.

FIG. 17 is a diagram illustrating an example of a process flow from thepre-search process P5 to the resolution determination process P3according to the seventh embodiment.

In this embodiment, first, an imaging environment is imaged by theimaging unit 30 while the imaging direction is adjusted by the pan andtilt mechanism of the camera device 20, and the captured image data D1of the imaging environment is acquired by the camera-side controller 34(S71 in FIG. 17). In the camera-side controller 34, the pre-searchprocess P5 is performed and the environmental frequency information D6is acquired (S72).

When the tracking imaging is started, the camera-side controller 34acquires the basic tracking information D4 at each imaging timing (see“the first timing” to the “fourth timing” described above) (S73), andperforms the resolution determination process P3 to acquire theresolution information D5 on the basis of the basic tracking informationD4 and the environmental frequency information D6 (S74).

OTHER MODIFICATION EXAMPLES

Each of the above-described functional configurations can be realizedarbitrary hardware, arbitrary software, or a combination of both and canbe realized by appropriately combining, for example, a centralprocessing unit (CPU), a volatile random access memory (RAM), anonvolatile memory such as an electrically erasable programmableread-only memory (EEPROM), and/or various operation programs such as anoperating system (OS) or an application program. Further, the presentinvention can be applied to a program that causes a computer to executea procedure of various processes regarding the image processing methodand the tracking imaging method in each unit of the terminal device 11and the camera device 20 described above, a computer-readable recordingmedium (non-transient tangible medium) having the program storedthereon, or a computer in which the program can be installed. Inparticular, each of the above-described processes in the terminal device11 may be executed on a dedicated application or may be executed on abrowser.

Further, a form of the terminal device 11 of the present invention isnot particularly limited, and the terminal device 11 may be constitutedby a mobile phone, a smart phone, a tablet terminal, personal digitalassistants (PDA), or a portable game machine.

EXPLANATION OF REFERENCES

10: tracking system

11: terminal device

12: housing

13: display

14: operation panel

15: touch panel

16: operation button

17: speaker

18: microphone

19: terminal camera

20: camera device

21: imaging optical system

22: device main body

23: pedestal

24: dome cover

25: holding unit

26: gear

30: imaging unit

32: imaging direction adjustment unit

34: camera-side controller

36: camera-side communication unit

40: terminal-side controller

42: user interface

44: terminal-side communication unit

What is claimed is:
 1. A tracking system, comprising: a camera deviceincluding an imaging unit that acquires captured image data, an imagingdirection adjustment unit that adjusts an imaging direction of theimaging unit, a camera-side controller that controls the imagingdirection adjustment unit on the basis of target information indicatinga position of a tracking target, and a camera-side communication unitthat is connected to the camera-side controller; and a terminal deviceincluding a terminal-side communication unit that is able to communicatewith the camera-side communication unit, a terminal-side controller thatis connected to the terminal-side communication unit, and a userinterface that is connected to the terminal-side controller, wherein atleast one of the camera-side controller or the terminal-side controllerperforms a tracking image generation process of generating trackingimage data from the captured image data, at least one of the camera-sidecontroller or the terminal-side controller performs a trackingcalculation process of acquiring the target information on the basis ofthe tracking image data, the tracking image data is data of an imageincluding a high-resolution region with relatively higher resolution anda low-resolution region with relatively lower resolution, in thetracking image generation process, basic tracking information includinginformation on at least one of a size or a speed of the tracking targetis acquired, the high-resolution region and the low-resolution regionare determined on the basis of the basic tracking information, and thetracking image data is generated, and the tracking image generationprocess and the tracking calculation process are performed in parallel.2. The tracking system according to claim 1, wherein at least a portionof the tracking calculation process of acquiring the target informationindicating the position of the tracking target at a first timing on thebasis of the tracking image data generated from the captured image dataacquired at the first timing is performed simultaneously with at least aportion of the tracking image generation process of generating thetracking image data that is used to acquire the target informationindicating the position of the tracking target at a second timing afterthe first timing, from the captured image data acquired at the secondtiming.
 3. The tracking system according to claim 1, wherein thetracking image generation process and the tracking calculation processare performed in one of the camera-side controller and the terminal-sidecontroller.
 4. The tracking system according to claim 1, wherein one ofthe tracking image generation process and the tracking calculationprocess is performed in one of the camera-side controller and theterminal-side controller, and the other of the tracking image generationprocess and the tracking calculation process is performed in the otherof the camera-side controller and the terminal-side controller.
 5. Thetracking system according to claim 1, wherein the tracking calculationprocess is performed by both of the camera-side controller and theterminal-side controller, and a controller having a relatively higherprocessing capability between the camera-side controller and theterminal-side controller performs the tracking calculation process ofthe high-resolution region, and a controller having a relatively lowerprocessing capability between the camera-side controller and theterminal-side controller performs the tracking calculation process ofthe low-resolution region.
 6. The tracking system according to claim 1,wherein at least one of the camera-side controller or the terminal-sidecontroller performs a pre-search process of acquiring a spatialfrequency of the captured image data of at least a portion of theimaging range of the camera device, and in the tracking image generationprocess, resolution of the tracking image data is determined on thebasis of the spatial frequency acquired through the pre-search process.7. The tracking system according to claim 6, wherein in the trackingimage generation process, in a case where the spatial frequency acquiredthrough the pre-search process is relatively low, resolution of thetracking image data is set to be lower than in a case where the spatialfrequency is relatively high.
 8. A terminal device that is connectableto a camera device including an imaging unit that acquires capturedimage data, an imaging direction adjustment unit that adjusts an imagingdirection of the imaging unit, a camera-side controller that controlsthe imaging direction adjustment unit on the basis of target informationindicating a position of a tracking target, and a camera-sidecommunication unit that is connected to the camera-side controller, theterminal device comprising: a terminal-side communication unit that isable to communicate with the camera-side communication unit; aterminal-side controller that is connected to the terminal-sidecommunication unit; and a user interface connected to the terminal-sidecontroller, wherein at least one of the camera-side controller or theterminal-side controller performs a tracking image generation process ofgenerating tracking image data from the captured image data, theterminal-side controller performs a tracking calculation process ofacquiring the target information on the basis of the tracking imagedata, and transmits the target information to the camera-side controllervia the terminal-side communication unit and the camera-sidecommunication unit, the tracking image data is data of an imageincluding a high-resolution region with relatively higher resolution anda low-resolution region with relatively lower resolution, in thetracking image generation process, basic tracking information includinginformation on at least one of a size or a speed of the tracking targetis acquired, the high-resolution region and the low-resolution regionare determined on the basis of the basic tracking information, and thetracking image data is generated, and the tracking image generationprocess and the tracking calculation process are performed in parallel.9. A camera device connectable to a terminal device including aterminal-side communication unit, a terminal-side controller that isconnected to the terminal-side communication unit, and a user interfacethat is connected to the terminal-side controller, the camera devicecomprising: an imaging unit that acquires the captured image data; animaging direction adjustment unit that adjusts an imaging direction ofthe imaging unit; and a camera-side controller that controls the imagingdirection adjustment unit on the basis of target information indicatinga position of the tracking target, the camera-side controller performinga tracking image generation process of generating tracking image datafrom the captured image data and a tracking calculation process ofacquiring the target information on the basis of the tracking image datain parallel, wherein the tracking image data is data of an imageincluding a high-resolution region with relatively higher resolution anda low-resolution region with relatively lower resolution, and in thetracking image generation process, basic tracking information includinginformation on at least one of a size or a speed of the tracking targetis acquired, the high-resolution region and the low-resolution regionare determined on the basis of the basic tracking information, and thetracking image data is generated.
 10. A tracking imaging method of atracking system comprising a camera device including an imaging unitthat acquires captured image data, an imaging direction adjustment unitthat adjusts an imaging direction of the imaging unit, a camera-sidecontroller that controls the imaging direction adjustment unit on thebasis of target information indicating a position of a tracking target,and a camera-side communication unit that is connected to thecamera-side controller, and a terminal device including a terminal-sidecommunication unit that is able to communicate with the camera-sidecommunication unit, a terminal-side controller that is connected to theterminal-side communication unit, and a user interface that is connectedto the terminal-side controller, the tracking imaging method comprising:a step of performing a tracking image generation process of generatingtracking image data from the captured image data by at least one of thecamera-side controller or the terminal-side controller; and a step ofperforming a tracking calculation process of acquiring the targetinformation on the basis of the tracking image data by at least one ofthe camera-side controller or the terminal-side controller, wherein thetracking image data is data of an image including a high-resolutionregion with relatively higher resolution and a low-resolution regionwith relatively lower resolution, in the tracking image generationprocess, basic tracking information including information on at leastone of a size or a speed of the tracking target is acquired, thehigh-resolution region and the low-resolution region are determined onthe basis of the basic tracking information, and the tracking image datais generated, and the tracking image generation process and the trackingcalculation process are performed in parallel.
 11. A computer-readablenon-transitory tangible medium having a program recorded thereon, theprogram for controlling a tracking system comprising a camera deviceincluding an imaging unit that acquires captured image data, an imagingdirection adjustment unit that adjusts an imaging direction of theimaging unit, a camera-side controller that controls the imagingdirection adjustment unit on the basis of target information indicatinga position of a tracking target, and a camera-side communication unitthat is connected to the camera-side controller, and a terminal deviceincluding a terminal-side communication unit that is able to communicatewith the camera-side communication unit, a terminal-side controller thatis connected to the terminal-side communication unit, and a userinterface that is connected to the terminal-side controller, the programcausing a computer to execute: a step of performing a tracking imagegeneration process of generating tracking image data from the capturedimage data by at least one of the camera-side controller or theterminal-side controller; and a step of performing a trackingcalculation process of acquiring the target information on the basis ofthe tracking image data by at least one of the camera-side controller orthe terminal-side controller, wherein the tracking image data is data ofan image including a high-resolution region with relatively higherresolution and a low-resolution region with relatively lower resolution,in the tracking image generation process, basic tracking informationincluding information on at least one of a size or a speed of thetracking target is acquired, the high-resolution region and thelow-resolution region are determined on the basis of the basic trackinginformation, and the tracking image data is generated, and the trackingimage generation process and the tracking calculation process areperformed in parallel.